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A  study  of  the  National  Severe  Storm  Laboratory’s  (NSSL’s)  Hail  Detection 
Algorithm  (HD A)  has  been  performed  for  over  20  different  multicell  storm  systems  that 
occurred  over  the  coastal  region  of  the  southern  Great  Plains.  Since  the  HDA  was 
developed  using  a  data  set  mostly  from  the  Central  Plains  where  many  of  the 
thunderstorms  are  supercellular,  this  study  addresses  the  regional  and  storm 
morphological  influences  on  the  algorithm  performance. 

To  enable  use  of  ground  validation  from  spotter  networks,  only  the  severe  hail 
category  was  studied.  While  the  NSSL  HDA  performed  quite  well  in  most  instances, 
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threshold,  and  the  Probability  of  Severe  Hail  (POSH)  function. 
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CHAPTER  I 

INTRODUCTION  AND  PREVIOUS  STUDIES 


a.  Introduction 

The  accuracy  and  timeliness  of  severe  weather  warnings  depend  on  both  the 
availability  of  observations  from  weather  radars  used  by  the  National  Weather  Service 
(NWS),  and  a  thorough  understanding  of  the  different  types  of  storm  structures  and 
environments  that  can  produce  severe  weather.  To  help  meet  this  challenge,  the  NWS 
has  recently  deployed  a  Doppler  weather  radar  network  across  Texas  and  much  of  the 
nation  (Klazura  and  Imy  1993),  where  each  site  is  responsible  for  severe  weather 
warnings  over  a  125  nautical  mile  range.  In  an  attempt  to  automate  the  detection  and 
interpretation  of  severe  weather  signatures  observed  by  Doppler  radar,  the  NWS  relies 
on  computer-based  algorithm  software,  some  of  which  has  been  developed  by  the 
National  Severe  Storms  Laboratory  (NSSL)  in  Norman,  Oklahoma.  This  software  is 
used  in  conjunction  with  the  WSR-88D  (Weather  Surveillance  Radar  -  1988  Doppler) 
to  examine  the  incoming  data  for  well-known,  classic  storm  structures  and  to  determine 
whether  or  not  the  storm  structure  is  severe.  One  such  algorithm  analyzes  convective 
cell  structure  to  determine  the  likelihood  of  severe  hail  (greater  than  0.75  inches  in 
diameter)  within  that  cell. 

Thunderstorms  that  produce  severe  hail  have  proven  to  be  very  costly  to  both 
densely  populated  areas  and  to  agricultural  interests  in  rural  areas.  The  NWS  office  in 
League  City,  Texas,  located  just  south  of  Houston,  is  responsible  for  providing  weather 
warnings  to  a  wide  range  of  counties  including  rural  areas  such  as  Houston,  Trinity, 
Chambers,  Madison,  Brazoria  and  Brazos  counties  to  densely  populated  counties  such 
as  Harris  and  Galveston  counties  (Figure  1). 


This  thesis  follows  the  style  of  the  Journal  of  Applied  Meteorology. 
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Figure  1.  The  NWS  League  City  county  warning  area.  The  black  dot  indicates 
the  NWS  office  at  League  City,  Texas.  The  black  circle  represents  the  200  kilometer 
range  of  the  WSR-88D  radar  located  at  the  NWS  office. 
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In  this  warning  region  alone  there  was  an  estimated  $105K  worth  of  hail  damage 
in  1995  (NOAA  1995),  $5.5M  worth  of  hail  damage  in  1996  (NOAA  1996),  and  close 
to  $600K  in  1997  (NOAA  1997).  The  ability  to  detect,  or  predict,  severe  hail  within 
thunderstorms  might  aid  in  minimizing  the  losses  due  to  this  damaging  weather 
phenomenon. 

The  conceptual  models  and  databases  used  to  develop  the  hail  detection 
algorithm  (HD A)  were  based  mostly  on  observations  in  the  Central  Great  Plains  of  the 
United  States  (Witt  et  al.  1998).  It  is  unclear  whether  these  data  sets  are  sufficient  for 
severe  weather  events  that  occur  along  the  Texas  Gulf  coast.  In  this  area,  the  weather 
regime  is  semitropical  and  is  mostly  dominated  by  warm  moist  air  that  flows  northward 
from  the  Gulf  of  Mexico.  Weather  events  initiated  in  this  area  have  a  more  complex 
convective  nature  where  the  environment  is  more  moist  and  the  wind  shear  is  weak. 
This  is  in  direct  contrast  to  the  Central  Great  Plains,  where  the  air  above  the  boundary 
layer  is  drier  and  the  wind  shear  is  more  pronounced. 

Because  regional  variability  in  storm  structure  was  known  to  exist  before  radar 
deployment,  NSSL  has  developed  the  WSR-88D  Algorithm  Testing  and  Display 
System  (WATADS).  This  system  allows  individual  NWS  offices  to  perform  local 
evaluations  of  the  algorithms  to  determine  the  optimum  threshold  values  for  their 
particular  warning  area.  This  is  made  possible  because  many  of  the  automated 
computer  algorithms  within  WATADS  contain  site  adaptable  parameters  that  can  be 
adjusted  to  improve  the  detection  of  severe  weather  in  regions  where  the  Central  Great 
Plains  conceptual  models  do  not  apply  (Crum  et  al.  1993).  However,  studies  to  provide 
guidance  to  the  local  NWS  offices  concerning  parameter  adjustments  have  been  limited 
by  lack  of  research  access  to  real-time  Doppler  data,  and  the  need  for  a  critical 
evaluation  of  the  modified  algorithm's  performance. 

Currently,  there  have  been  no  studies  directed  to  developing  a  suitable  set  of 
adaptable  parameters  for  the  Texas  Gulf  coast.  We  address  this  concern  here  by 
performing  an  analysis  of  Texas  Gulf  coast  storms  to  determine  the  optimal  threshold 
values  used  in  the  HDA.  By  testing  this  algorithm  in  a  natural  environment  that  is 
different  from  where  it  was  developed,  it  will  be  possible  to  provide  more  useful 
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guidance  to  the  NWS  offices  in  the  State  of  Texas  and  the  remainder  of  the  Gulf  coast 
region. 

The  focus  and  goal  of  this  study  will  be  two-fold.  The  first  goal  will  be  to  see 
how  the  hail  algorithm  performs  in  the  League  City  NWS  county  warning  area  using  the 
WATADS  software  along  with  recently  collected  Level  II  radar  and  local 
environmental  data.  The  second  goal  will  be  to  take  the  information  above,  and  try  to 
improve  the  hail  detection  and  warnings  over  this  area  by  tuning  the  hail  algorithm 
using  systematic  site-adaptable  parameter  adjustments. 

b.  Previous  studies 

The  original  WSR-88D  Hail  Algorithm  was  developed  by  Petrocchi  (1982). 
The  design  was  based  on  identification  of  the  structural  characteristics  of  typical  severe 
hailstorms  found  in  the  Southern  Plains  (Lemon  1978).  Lemon  examined  the  entire 
three-dimensional  reflectivity  structure  in  identifying  severe  hailstorms.  Using 
hailstorms  case  studies,  he  showed  that  the  height  of  a  given  reflectivity  value  was  more 
correlated  with  severe  hailstorms  than  peak  reflectivity  values  for  a  particular  storm. 
He  also  showed  that  the  position  of  the  highest  echo  top  relative  to  the  low-level 
reflectivity  core  was  strongly  correlated  with  whether  or  not  the  storm  will  produce  hail. 
Another  major  influence  in  hailstorm  development  was  the  extent  of  the  Weak  Echo 
Region  of  the  storm,  or  the  amount  of  overhang  of  the  mid-level  reflectivity  echo. 
From  these  findings,  several  hail  indicators  were  determined  to  be  the  most  useful  in 
representing  the  storm  structure  and  the  conditions  necessary  for  hailstorm  development 
(Smart  and  Alberty  1985).  Appendix  A  gives  a  detailed  description  of  these  findings. 

Early  testing  of  this  algorithm  in  the  Central  Plains  showed  good  performance 
(Petrocchi  1982;  Smart  and  Alberty  1985).  However,  subsequent  testing  by  Winston 
(1988)  over  the  Colorado  High  Plains  showed  relatively  poor  performance  by  the  hail 
algorithm  due,  in  large  part,  to  the  different  weather  regime  in  which  the  algorithm  was 
tested.  The  algorithm’s  ability  to  distinguish  hailstorms  from  non-hailstorms,  or  severe 
hailstorms  from  non-severe  hailstorms,  was  not  adequate. 
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Irrespective  of  its  apparent  lack  of  performance  in  other  weather  regimes,  the 
utility  of  the  hail  algorithm  was  not  of  much  use  to  NWS  forecasters  due  to  the  nature 
of  its  output.  Petrocchi’s  algorithm  was  only  optimized  to  detect  hail  of  size  1.25 
inches  or  greater.  Since  NWS  forecasters  were  required  to  provide  warnings  for  severe 
size  hail  being  >  0.75  inches,  they  needed  a  modified  algorithm  optimized  for  this  size 
and  not  the  hail  size  as  proposed  by  Petrocchi’s  algorithm. 

This  has  led  to  the  design  and  development  of  a  new  Hail  Detection  Algorithm 
(HD A)  for  the  WSR-88D  (Witt  1993).  Given  the  uncertainties  involved  in 
characterizing  hailstorms,  empirical  probability  functions  are  now  used.  Also,  in  place 
of  the  previous  labels,  the  new  algorithm  produces,  for  each  detected  storm  cell,  the 
following  information:  probability  of  hail  of  any  size  (POH),  probability  of  severe  hail 
(POSH),  and  maximum  expected  hail  size  (MEHS).  All  acronyms  used  during  this 
study  will  be  listed  in  Appendix  B. 
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CHAPTER  II 

NSSL  HAIL  DETECTION  ALGORITHM  DESIGN 


One  of  the  latest  algorithms  to  be  placed  into  WATADS  is  the  Hail  Detection 
Algorithm.  As  mentioned  above,  the  purpose  of  the  NSSL  HDA  is  to  provide  reliable 
estimates  of  the  probability  of  hail,  probability  of  severe  hail,  and  the  maximum 
expected  hail  size  for  each  storm  cell  identified  by  the  system.  The  HDA  is  composed 
of  two  sub-algorithms:  the  Hail  Core  Aloft  Algorithm  (HCAA)  (Witt  1990;  1993)  and 
the  Upper-Level  Divergence  Algorithm  (ULDA).  However,  since  this  study  will  only 
be  examining  an  algorithm  contained  in  the  HCAA,  the  ULDA  will  not  be  discussed. 

The  HCAA  currently  produces  all  of  the  HDA  output.  The  HCAA  estimates  the 
POH,  the  POSH  (hail  with  diameter  >  0.75  in),  and  the  MESH  for  each  detected  storm 
cell  (Witt  et  al.  1998).  This  algorithm  runs  in  conjunction  with  the  Storm  Cell 
Identification  and  Tracking  (SCIT)  algorithm  developed  at  NSSL  to  locate  and  track  the 
center  of  a  storm  cell.  This  algorithm  process  consists  of  basically  four  main  steps. 

The  first  step  begins  with  the  algorithm  searching  along  the  radials  for  data 
segments  that  contain  reflectivities  greater  than  a  specified  threshold.  Each  part  of  the 
ray  of  radar  data  that  exceeds  the  threshold  is  referred  to  as  a  storm  segment.  Figure  2 
shows  separate  storm  segments  with  different  threshold  values  along  three  adjacent 
radar  rays.  The  second  step  involves  combining  the  azimuthally  adjacent  segments  into 
2D-storm  components,  also  shown  in  Figure  2.  The  algorithm  then  compiles  the  newly 
computed  2D  information  throughout  all  the  volume  scans  and  turns  all  of  its  vertically 
adjacent  2D  components  into  “3D-storms”.  The  storm  cell  in  Figure  Al  illustrates  this 
process. 

Finally,  the  SCIT  algorithm  produces  output  on  the  storm’s  structural 
characteristics  and  displays  it  on  the  WATADS  radar  reflectivity  display.  An 
illustration  of  the  WATADS  display  is  given  on  Figure  3.  This  data  comes  from  21 
January  1998  and  shows  an  example  of  the  identification  of  storm  centers  by  the  SCIT 
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Figure  2.  This  image  represents  the  joining  of  three  adjacent  storm  segments 
into  a  2D-storm  component.  The  shading  denotes  a  40  dBz  threshold.  The  graduated 
lines  represent  azimuthal  radar  beams  at  a  fixed  elevation  angle.  Adapted  from  Johnson 
etal.  (1998). 


8 


Figure  3.  This  figure  shows  the  WATADS  radar  display  for  Walker  County  on 
21  January  1998.  The  boxed  values  are  the  storm  center  values  that  the  SCIT  algorithm 
assigns  to  each  storm  cell.  The  dark-shaded  boxes  depict  cells  with  POSH  values  > 
50%  (ex.  #47).  The  light-shaded  boxes  depict  cells  with  POSH  values  between  20% 
and  40%  (ex.  #31).  The  medium-shaded  boxes  depict  cell  with  POSH  values  <  20% 
(ex.  #8). 
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algorithm.  The  following  paragraphs  explain  how  the  POH  and  POSH  algorithms 
work,  and  elucidate  all  the  parameters  defined  within  them. 

a.  POH 

To  determine  the  POH  of  any  size  in  a  particular  storm  cell,  the  HDA  employs 
an  adaptation  of  a  Waldvogel  et  al.  (1979)  relation  between  the  height  (km)  of  the  45 
dBZ  echo  (H45)  and  the  height  (km)  of  the  freezing  level  (Ho)  in  determining  the 
probability  of  hail  reaching  the  ground.  This  is  illustrated  in  Figure  4,  which  shows  the 
relationship  between  POH  and  the  difference  between  H45  and  Ho.  The  greater  the 
distance  between  the  height  of  the  45  dBZ  echo  top  and  the  height  of  the  freezing  level, 
the  greater  the  POH.  Since  the  algorithm's  output  has  been  converted  to  probabilities, 
Figure  4  shows  percentages  in  increments  of  10%.  However,  the  values  from  this 
relation  were  derived  using  only  hail  storm  data  from  a  High  Plains  environment,  so 
caution  should  be  used  when  applying  this  to  other  environments. 

b.  POSH 

The  POSH  parameter  determines  the  probability  of  severe  hail  (diameter  >  0.75  inch). 
This  probability  is  obtained  with  the  following  equation: 

POSH  =  29 In  +50  W 

WT 

where  SHI  is  the  Severe  Hail  Index  and  WT  is  the  Warning  Threshold  function. 

SHI  was  developed  as  the  primary  predictor  for  severe  size  hail.  It  uses  an 
approach  similar  to  the  vertical  integration  of  liquid  (VIL),  estimated  from  a 
reflectivity-liquid  relationship,  with  a  few  improvements.  The  first  improvement 
involves  moving  from  a  grid-based  algorithm  to  a  storm  cell-based  algorithm,  using 
output  from  the  SCIT  algorithm  discussed  earlier.  The  advantage  of  a  storm  cell-based 


Probability  of  Hail  (%) 


Figure  4.  Graph  showing  relationship  between  POH  and  the 
height  of  the  freezing  level  (Ho)  and  the  height  of  the  45  dBZ  reflectivity 
core.  Derived  from  Waldvogel  et  al.  (1979). 


11 


system  is  the  problem  associated  with  having  a  hail  core  cross  a  grid  boundary,  which 
occurs  frequently,  is  eliminated.  However,  one  disadvantage  to  this  method  is  that  if  an 
error  occurs  in  the  cell  identification  process,  this  may  cause  an  error  in  the  HDA 
output. 

The  second  improvement  involves  using  a  reflectivity-to-hail  relation,  based  on 
Waldvogel  et  al.  (1978a,b).  Waldvogel  derived  a  relationship  between  the  radar 
reflectivity  factor  (Z)  and  the  flux  of  kinetic  energy  ( E )  by  analyzing  hail  size 
distributions  measured  with  hail  spectrometers.  This  reflectivity-to-hail  relation  is 
similar  in  principle  to  the  reflectivity-to-rainrate  (Z-R)  relation,  used  widely  to  estimate 
rainfall  amounts,  which  analyzes  variations  of  (Z-R)  with  raindrop  disdrometers  to 
represent  an  ideal  raingage-radar  combination  (Waldvogel  et  al.  1978a).  Here, 
Waldvogel  discovered  the  hail  spectrometer  represented  an  ideal  hailpad-radar 
combination.  He  also  discovered  that  the  variations  in  the  Z-  E  relationship  throughout 
a  wide  variety  of  storms  tested  were  very  small.  Waldvogel  developed  a  semi-empirical 
formula  that  could  be  computed  by  assuming  Rayleigh  scattering  and  an  exponential 
size  distribution  for  the  hailstone  spectra  (Waldvogel  et  al.  1978a,b;  Federer  et  al. 
1986).  Witt  et  al.  (1998)  modified  the  Hailfall  Kinetic  Energy  equation  by  adding  a 
weighting  function  such  that: 


E  =  5.0  x  10^  x  100084Z  W(Z ) 


(2) 


where  E  has  units  of  J  m'2  s'1.  Z  is  taken  to  be  the  logarithmic  value  of  the  radar 
reflectivity  factor  (measured  in  dBZ  -  explained  in  Appendix  C)  and  W(Z)  is  the 
weighting  function  defined  as: 


r 


< 


0 

z  -  z, 


for  Z  <  Zl 
for  Zl  <  Z  <  Zu 


W(Z)  = 


Zu-Zl 

1 


for  Z  >  Zu 


(3) 
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W(Z)  is  used  to  define  a  transition  zone  between  rain  and  hail  reflectivities.  Zu  and  Zl 
are  reflectivity  upper  and  lower  boundaries  and  are  set  by  default  as  Zu  =  50  dBZ  and 
Zl  =  40  dBZ.  This  Z-  E  relationship  uses  only  the  higher  reflectivities  typically 
associated  with  hail  and  filters  out  most  of  the  lower  reflectivities  that  are  typically 
associated  with  liquid  water. 

The  last  improvement  within  the  SHI  involves  the  temperature  range  where  hail 
is  most  likely  to  form.  Because  hail  growth  will  only  occur  at  temperatures  <  0°C,  and 
severe  hail  growth  will  only  occur  at  temperatures  <  -20°C  or  colder  (English  1973; 
Browning  1977;  Nelson  1983;  Miller  et  al.  1988),  a  unitless,  temperature-weighted, 
vertically  integrated  function  Wt(H)  is  used  within  the  SHI  function.  It  is  defined  as: 


Wt(H)  = 


r 

0  for  H  <  H0 

J  H  -  Hn  forH0<H<Hm20 

Hm20  “  Ho 

1  for  H  >  Hm2o 


(4) 


Where  H  is  the  height  above  radar  level  (ARL),  Ho  is  the  height  (ARL)  of  the 
environmental  melting  level,  and  H^o  is  the  height  (ARL)  of  the  -20°C  environmental 
temperature.  All  heights  are  in  kilometers.  This  function  weighs  the  temperatures 
below  -20°C  the  strongest,  gives  temperatures  between  0°C  and  -20°C  a  small  factor  in 
the  equation,  but  completely  negates  any  values  warmer  than  0°C.  Both  Ho  and  Hm2o 
can  be  determined  from  a  nearby  upper-air  sounding. 

Finally,  using  all  of  the  above  parameters  we  can  obtain  what  is  known  as  the 
Severe  Hail  Index,  which  has  units  of  J  m'1  s'1. 

ht 

SHI  =  0.ljwT(H)EdH 

»o 


(5) 
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where  Ht  is  the  height  (in  km)  of  the  highest  echo  of  the  storm  cell.  In  the  HDA,  SHI  is 
calculated  using  information  from  the  2D-storm  components  for  the  cell  being  analyzed, 
with  at  least  two  components  required  for  calculation.  E  is  calculated  using  the 
maximum  reflectivity  value  for  each  2D-storm  component.  This  value  is  then  applied 
throughout  the  entire  vertical  depth  of  the  2D-storm  component,  or  the  entire  “3D- 
storm”. 

From  the  Witt  et  al.  study  (1998),  the  SHI  underwent  developmental  testing  at 
NSSL,  where  they  used  data  from  115  storm  reports  that  included  severe-size  hail. 
From  these  reports,  it  was  discovered  that  the  optimum  value  of  WT  was  highly 
correlated  to  the  melting  level  of  that  day.  From  these  results,  a  simple  Warning 
Threshold  Selection  Model  (WTSM)  was  created,  and  is  defined  as: 

WT  =  51.5Hq  -121  (6) 

where  WT  has  units  in  J  m'V1  and  Ho  has  units  in  kilometers. 

Both  of  these  parameters  proved  fairly  successful  in  testing  done  by  NSSL,  with 
over  Critical  Success  Index  (CSI)  values  of  greater  than  40%  (Witt  et  al.  1998).  The 
next  and  final  step  was  to  create  an  overall  probability  function  that  incorporated  the 
two  parameters.  Since  the  data  set  was  fairly  small,  a  simple  probability  function  was 
used. 

This  has  led  to  the  POSH  equation  (1).  The  POSH  parameter  is  measured  in 
percentiles  that  are  rounded  of  to  the  nearest  10%,  in  order  to  avoid  conveying  an 
unrealistic  degree  of  precision.  POSH  values  <  0  are  set  to  0,  while  POSH  values  >100 
are  set  to  100.  Note  that  the  probability  of  severe  hail  is  50%  when  the  value  of  the 
severe  hail  index  is  equal  to  that  of  the  warning  threshold. 
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CHAPTER  III 

DATA  AND  METHOD  OF  ANALYSIS 


a.  Data 

The  convective  events  used  for  this  study  occurred  within  the  League  City 
forecasting  area  of  responsibility.  The  hailstorms  chosen  for  this  data  set  occurred  both 
between  the  years  1994  and  1997,  and  mostly  during  the  months  of  March  through 
August.  Based  on  Storm  Data  reports  from  NOAA,  these  months  were  the  optimum 
times  for  severe  hail  to  form  over  this  warning  area.  The  data  set  included  level  II  radar 
data  archived  from  the  WSR-88D  Doppler  radar  from  the  NWS  office  in  League  City, 
Texas.  Environmental  data  were  provided  by  upper-air  soundings  from  Corpus  Christi, 
Texas  and  Lake  Charles,  Louisiana;  and  hail  verification  reports  were  provided  by 
Storm  Data. 

Level  II  radar  data  were  the  main  data  source  for  this  study.  The  hail  days  were 
carefully  chosen  in  conjunction  with  the  storm  data  to  find  hail  days  around  the  Houston 
area  with  the  maximum  amount  of  hail  verification.  To  examine  the  thermodynamics  of 
the  environment,  soundings  were  obtained  from  the  National  Climatic  Data  Center  and 
were  then  ingested  into  WATADS. 

There  were  120  total  hail  reports  used  for  this  study.  They  can  be  found  on 
Table  1.  Since  this  study  will  be  examining  the  effects  of  different  storm  types  on  the 
hail  algorithm,  the  data  were  divided  between  convective  events  forced  by  seabreeze 
fronts  and  those  forced  by  continental  fronts  or  troughs.  Only  15  of  the  reports  were 
associated  with  seabreeze  fronts.  Two-thirds  of  the  events  took  place  during  the  spring- 
summer  seasons  (March  -  August).  The  remainder  of  the  reports  was  associated  with 
frontally  forced  deep  convection  during  the  winter  season. 

Storm  data  that  was  located  within  the  radar’s  cone  of  silence,  less  than  a  30 
kilometer  radius,  or  at  ranges  greater  than  200  kilometers  were  not  analyzed.  This 
exclusion  is  due  to  the  radar’s  inability  to  give  meaningful  data  at  these  ranges. 
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Table  1.  Complete  listing  of  hail  reports  used  for  this  study.  The  data  is  broken  down 
into  hail  reports  from  convection  due  to  Seabreeze  fronts  and  convection  due  to  cold 
fronts  or  troughs  advecting  into  the  warning  area  from  the  north/northwest.  From  radar 
archives  at  the  NWS  office  in  League  City,  Texas. 


SEABREEZE  DATA 

Number  of  Hail 

Date 

Description 

Reports 

02  Jun  96 

Outflow  Boundary 

3 

03  Jun  96 

Outflow  Boundary 

2 

04  Jun  96 

Outflow  Boundary 

7 

21  May  97 

Outflow  Boundary 

1 

02  Sep  97 

Outflow  Boundary 

2 

Total  Seabreeze: 

15 

CONTINENTAL  DATA 

Number  of  Hail 

Date 

Description 

Report 

05  Apr  94 

Cold  Front  /  Severe  Thunderstorms  /  Hail 

8 

15  Apr  94 

Cold  front 

10 

02  May  94 

Bow  Echo 

11 

11  May  95 

Supercells  /  Hail 

10 

03  Nov  95 

Cold  Front  /  Heavy  Thunderstorms  /  Hail  /  Wind 

8 

05  Apr  96 

Severe  Thunderstorms  /  Flooding 

9 

25  Apr  97 

Trof  /  Thunderstorms  /  Small  Hail 

5 

27  Apr  97 

Trof  /  Thunderstorms  /  Small  Hail 

6 

28  May  97 

Thunderstorms  /  Hail  /  Heavy  Rain 

2 

09  Sep  97 

Cold  Front  /  Hail 

6 

21  Jan  98 

Cold  Front  /  Hail  /  Flooding 

8 

22  Jan  98 

Cold  Front  /  Hail  /  Flooding 

15 

1 1  Apr  97  Trof  /  Thunderstorms  7 

Total  Continental:  105 

TOTAL  OF  ALL  HAIL  REPORTS:  120 
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b.  Method  of  analysis 

1)  SELECTING  HAIL  DAYS 

Eighteen  hail  days  were  chosen  from  the  archived  level  II  radar  data.  There 
were  no  “non-hail  days”  chosen  as  part  of  this  study.  The  addition  of  non-hail  days  into 
the  data  set  would  introduce  more  false  alarms  into  the  scoring,  thereby  indicating 
where  the  algorithm  might  be  over-forecasting.  While  this  is  a  valuable  objective  of 
this  study,  it  can  be  accomplished  without  the  inclusion  of  non-hail  days  based  on  two 
data-spacing  factors:  location  and  time. 

The  first  factor  deals  with  the  location  of  the  hail  event.  There  are  cases  within 
this  data  set  where  two  different  weather  events  are  affecting  different  parts  of  the 
warning  area.  For  instance,  radar  data  from  one  hail  day  includes  hail  verification  from 
a  storm  event  in  the  extreme  northern  counties.  During  the  same  period,  storm  cells 
develop  in  the  extreme  southern  counties,  yet  are  not  verified  by  Storm  Data  as  being 
hail-producing  storms.  However,  the  algorithm  scoring  from  the  southern  county 
storms  are  included  as  part  of  the  same  hail  verification  in  the  north.  These  cases 
illustrate  hail  days  coupled  with  non-hail  occurrences,  and  can  be  included  in  the 
scoring  to  increase  the  false  alarm  rate,  thereby  providing  more  realistic  results. 

The  second  factor  deals  with  time  lapses  between  hail  events.  Included  in  the 
data  set  are  days  where  the  lapses  between  hail  verification  times  exceed  four  hours. 
Yet,  the  algorithm  is  still  verifying  hailstorms,  which  will  all  turn  out  to  be  false  alarms 
in  the  algorithm’s  final  output.  These  time  lapses  warrant  enough  consideration  to  be 
thought  of  as  non-hail  days,  given  the  quantity  of  volume  scans  between  the  verified 
hail  occurrences  within  the  data  set. 

2)  ALGORITHM  OUTPUT 

The  next  step  is  running  the  hail  algorithm,  which  uses  level  II  radar  data  that 
generates  an  output  file  for  each  day.  This  file  consists  of  the  locations  and  SHI  values 
of  all  storm  cells  detected  by  the  SCIT  algorithm,  and  is  organized  by  volume  scan 
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number.  However,  the  detection  of  numerous  smaller-scale  cells  within  a  larger,  multi- 
celled  storm  can  overload  the  output  file  and  give  muddled  results.  Therefore,  an 
adaptable  parameter  within  WAT  ADS  allows  the  user  to  select  a  minimum  separation 
distance  between  detected  cells.  This  allows  only  the  dominant  cell  within  a  multi- 
celled  storm  to  be  detected  by  the  SCIT  algorithm.  This  parameter  is  set  to  10  km  for 
this  study. 

3)  HAIL-TRUTH  FILE  AND  TIME  WINDOWS 

After  the  algorithms  are  run  using  the  radar  data,  the  next  step  is  to  set  up  a  hail- 
truth  file  for  each  day  studied.  This  file  relates  the  actual  hail  reports  to  the  storm  cells 
observed  by  the  algorithm.  It  accomplishes  this  by  recording  the  time  and  location  of 
the  cell  that  produced  the  verified  hail  report  for  each  volume  scan.  Since  the  only  hail 
verification  in  this  study  is  via  Storm  Data,  and  since  its  precision  in  the  spatial  and 
temporal  is  not  the  best,  a  time  window  is  introduced  into  the  hail-truth  file  to  reduce 
this  inaccuracy.  This  was  done  by  recording  the  time  and  location  of  a  cell  that  is 
associated  with  a  hail  report  both  before  and  after  the  time  of  the  report.  For  this  study 
a  time  window  of  20  minutes  was  used  for  each  hail  report  recorded  in  the  hail-truth 
file.  This  was  based  on  a  time-window  methodology  used  by  Witt  et  al.  (1998)  in  his 
analysis  of  the  hail  algorithm. 

Witt’s  time  window  was  20  minutes  in  length  and  looked  15  minutes  prior  and  5 
minutes  after  the  recorded  hail  report.  This  time  length  was  chosen  based  on  the  time 
taken  for  large  hail  to  fall  out  of  a  storm  (Changnon  1968).  The  actual  time  is  10 
minutes,  but  a  five-minute  buffer  was  added  to  both  sides  of  the  window  to  account  for 
synchronization  errors  between  the  radar  and  the  observed  hail  report.  Table  2  shows 
an  example  of  a  hail-truth  file  used  for  this  study,  illustrating  the  20-minute  time 
window. 
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4)  ALGORITHM  SCORING  AND  STATISTICS 

The  scoring  of  the  algorithm  is  a  three-part  process.  First,  a  fortran  program  is 
used  to  compare  the  hail-truth  file  with  the  output  file.  This  program  takes  each  line  of 
the  hail-truth  file  and  matches  it  to  a  corresponding  line  (by  time  and  location)  in  the 
output  file.  When  running  the  program,  the  user  must  input  a  wide  range  of  warning 
threshold  (WT)  values.  Depending  on  certain  threshold  values,  the  program  detects  and 
tabulates  a  hit,  miss,  or  false  alarm  for  severe  hail,  and  places  this  information  in  a 
separate  output  file.  The  next  step  in  the  process  involves  running  another  fortran 
program  on  the  newly  formed  output  file  that  finds  and  eliminates  duplicate  lines  in  the 
output. 

After  the  first  two  parts  are  successfully  completed,  the  final  part  of  the  scoring 
involves  computing  the  statistical  performance  results  of  the  algorithm.  The  algorithms 
are  evaluated  using  the  POD  (Probability  Of  Detection),  FAR  (False  Alarm  Ratio),  and 
CSI  (Critical  Success  Index)  statistics  (Wilks  1995).  Table  3  defines  these  values. 

5)  POSH  RELIABILITY 

After  the  algorithm  scoring  is  completed,  the  next  step  is  to  look  at  the  overall 
reliability  of  the  algorithm.  The  reliability  of  this  algorithm  is  determined  by  comparing 
the  POSH  values  with  the  observed  relative  frequency  (ORF)  of  each  hail  event.  For 
example,  suppose  one  forecasts  rain  for  a  particular  region  for  10  days  and  all  of  the 
forecasts  call  for  a  30%  probability  of  rain.  Now,  suppose  that  observations  show  that 
rain  occurred  only  2  out  of  the  10  days.  The  ORF  would  be  the  frequency  of  the  actual 
observed  event,  or  20%.  This  demonstrates  the  reliability  of  the  forecasts  and  shows 
that  the  predictions  are  higher  (30%)  than  the  observations,  thereby  leading  to  a  small 
over-forecasting  bias. 

Calculating  the  reliability  of  the  POSH  algorithm  is  slightly  more  complicated. 
As  stated  before,  the  hail  observations  in  Storm  Data  are  not  very  dependable  and  only 
represent  a  small  portion  of  the  hail  event.  Therefore,  it  is  necessary  to  use  the  time- 
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Table  2.  The  partial  hail-truth  file  for  hail  reported  on  03  November  1995.  The  value 
on  the  first  line  is  the  number  of  hail  reports  for  that  day.  The  second  line  indicates  the 
size  of  hail  and  the  time  in  which  it  was  reported.  The  following  lines  indicate  the 
storm  locations  for  the  volume  scans  ~15  minutes  before  the  report  and  ~5  minutes  after 
the  report,  giving  a  20-minute  time  window  for  verification. 


9  [#  of  hail  reports] 

19  [size  (mm)  of  hail] 
Azimuth  (°) 

0450  [time  (Z)  of  report] 
Range  (km) 

Volume  Scan  Time  (Z) 

354 

156 

0433 

355 

159 

0439 

356 

162 

0445 

358 

166 

0451 

359 

169 

0457 

Table  3.  2X2  contingency  table  of  forecast  versus  observed  states,  used  to  determine 
statistical  skill  scores  for  Y  (yes)  /  N  (no)  predictions  of  an  event.  POD  is  the  likelihood 
that  the  event  would  be  forecast,  given  that  it  occurred.  FAR  is  the  proportion  of 
forecast  events  that  fail  to  materialize.  CSI  is  the  number  of  correct  “yes”  forecasts 
divided  by  the  total  number  of  occasions  on  which  that  event  was  forecast  and/or 
observed. 


Forecast  Event 

Forecast  No  Event 

Total 

Observed  Event 

YY  (Hit) 

YN  (Miss) 

YY+YN 

Observed  No  Event 

NY  (False  Alarm) 

NN 

NY+NN 

Total 

YY+NY 

YN+NN 

Where: 


POD  = 


YY 

YY  +  YN 


FAR  = 


NY 

YY  +  NY 


CSI  = 


YY 


YY  +  NY+YN 


Observed  Relative  Frequency  (%) 
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POSH  (%) 


Figure  5.  Reliability  diagram  for  the  probability  of  severe  hail  for 
10  days  of  radar  data  from  Oklahoma  and  Florida.  The  numbers  adjacent 
to  the  plotted  points  indicate  the  number  of  forecasts  for  a  particular 
POSH  value.  The  x-axis  represent  the  forecasted  POSH  values,  while  the 
y-axis  represent  the  observed  relative  frequency  of  the  severe  hail 
occurrence.  The  diagonal  line  represents  perfect  reliability.  Adapted  from 
Witt  et  al.  (1998). 
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window  methodology  to  increase  the  number  of  hail  reports  (observations)  that  will  be 
associated  with  a  specific  hail  prediction.  So,  in  order  to  calculate  the  ORF  values  for 
POSH,  it’s  simply  a  matter  of  determining  the  number  of  times,  for  each  POSH 
category,  that  these  observations  occur  within  the  time  window  compared  to  the  total 
number  of  predictions.  For  example,  if  one  analyzes  a  day  that  has  a  total  of  100  POSH 
predictions  at  40%,  and  25  of  these  predictions  occur  within  the  time  window  allotted 
for  each  hail  report,  then  the  ORF  will  be  25%  for  the  40%  POSH  category.  Therefore, 
the  reliability  of  the  POSH  algorithm  shows  an  over-forecasting  bias  at  POSH=40%. 
Figure  5  shows  an  example  of  a  POSH  reliability  diagram  for  a  NSSL  hail  study  using 
10  days  of  radar  data  from  Oklahoma  and  Florida.  Notice  the  over-forecasting  bias 
from  POSH  values  from  10%  through  60%. 


6)  OPTIMIZING  THE  WTSM 

One  of  the  inputs  to  the  POSH  equation  is  the  warning  threshold  (WT)  which 
depends  of  the  height  of  the  melting  layer  according  to  the  WTSM  (Eq.  6),  where  WT  is 
the  Warning  Threshold  and  Ho  is  the  height  (in  km)  of  the  melting  layer.  The  WTSM 
was  derived  primarily  from  springtime  hail  events  in  the  Southern  Plains  and  Florida 
(Witt  et  al.  1998).  Since  the  WTSM  was  developed  in  a  different  region,  an  adjustment 
to  the  equation  for  different  environments  may  be  required. 

For  each  hail  event,  there  exists  an  optimum  warning  threshold  that  produces  the 
highest  CSI.  This  can  be  plotted  against  the  melting  layer  of  the  day  to  produce  a  new, 
more  accurate  WTSM  for  the  League  City  area.  A  graphical  example  can  be  found  on 
Figure  6.  This  comes  from  the  same  NSSL  hail  study  used  to  show  the  reliability  graph 
on  Figure  5.  The  parameters  within  Eq.  (6)  that  can  be  adjusted  are  the  y-intercept 
value  (-121)  and  the  slope  value  (57.5).  Local  soundings  from  Corpus  Christie,  TX  and 
Lake  Charles,  LA  will  provide  the  necessary  melting  layer  values. 

The  optimum  warning  threshold  is  determined  for  each  hail  day  when  scoring 
the  algorithm.  This  is  done  by  entering  a  wide  range  of  warning  threshold  (WT)  values 
into  the  scoring  algorithm,  and  choosing  the  one  WT  value  that  leads  to  the  highest  CSI. 


Figure  6.  This  figure  shows  the  optimum  warning  threshold  (black 
line)  as  a  function  of  the  melting  layer  from  the  initial  HDA  tests  made  by 
NSSL.  The  solid  circles  correspond  to  the  highest  CSI.  The  vertical  bars 
represent  the  range  of  warning  threshold  values  within  5  percentage  points 
of  the  maximum  value.  Adapted  from  Witt  et  al.  (1998). 
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Recalling  equation  (1)  shows  that  the  probability  equation  for  POSH  involves  a 
direct  ratio  between  the  SHI  value  (computed  for  every  storm  cell  and  volume  scan)  and 
the  WT  value.  The  scoring  algorithm  declares  a  “yes”  forecast  for  severe  hail  if  the  SHI 
value  is  greater  than  or  equal  to  the  WT  value;  otherwise  a  “no”  forecast  for  severe  hail 
is  made.  Depending  on  the  WT  value  given,  the  POD,  FAR  and  CSI  values  can  change. 
Table  4  shows  an  example  of  the  algorithm’s  performance  results  for  09  September 
1997.  Here,  the  highest  CSI  value  of  35  corresponds  to  a  WT  value  of  120.  Given  the 
melting  level  for  the  day,  this  WT  value  can  now  be  plotted  as  one  of  18  points. 
Finally,  a  new  WTSM  line  can  be  determined  by  ascertaining  a  “best  fit”  line  from  all 
data  points. 

7)  POPULATION  DENSITY  CONSIDERATIONS 

The  accuracy  of  this  algorithm’s  performance  is  dependent  upon  accurate  and 
timely  observations.  Therefore,  when  looking  to  adapt  this  algorithm  to  a  specific 
location,  strong  consideration  should  be  made  as  to  whether  or  not  the  population 
density  has  any  effect  on  the  reliability  of  the  algorithm. 

Wyatt  and  Witt’s  (1997)  study  showed  that  in  some  locations  the  reporting  of 
hail  is  better  in  areas  of  high  population  density  than  in  areas  where  there  are  few 
people  to  report  it.  Therefore,  the  analysis  of  a  hail  detection  algorithm’s  performance 
as  a  function  of  population  density  may  effectively  reduce  the  number  of  “false  alarms” 
generated  by  the  algorithm,  thereby  increasing  its  CSI  score.  This  finding  is  interesting 
for  this  study  due  the  fact  that  Harris  County,  which  includes  the  city  of  Houston,  has  an 
average  population  density  of  630  persons  per  km2.  This  is  in  direct  contrast  to  both 
Grimes  and  San  Jacinto  counties,  adjacent  to  Harris  County,  whose  average  population 
densities  are  equivalent  to  approximately  10  persons  per  km2. 

The  population  density  program  works  in  the  same  way  as  the  original  algorithm 
scoring  program,  with  some  minor  differences.  The  population  density  program 


24 


Table  4.  Performance  results  from  the  hail  algorithm  for  09  September  1 997  using  the 
20-minute  time  window.  WT  is  the  warning  threshold.  Here,  the  highest  CSI  value 
(35)  corresponds  to  a  warning  threshold  value  of  120.  The  POD,  FAR,  and  CSI  values 
are  all  in  percent. 


WT 

Hits 

Misses 

False 

Alarms 

POD  (%) 

FAR  (%) 

CSI  (%) 

80 

16 

7 

29 

70 

64 

31 

90 

15 

8 

25 

65 

63 

31 

100 

15 

8 

24 

65 

62 

32 

110 

14 

9 

22 

61 

61 

31 

120 

14 

9 

17 

61 

55 

35 

130 

11 

12 

16 

48 

59 

28 

140 

11 

12 

13 

48 

54 

31 

150 

11 

12 

12 

48 

52 

31 

160 

10 

13 

10 

43 

50 

30 

170 

9 

14 

9 

39 

50 

28 
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includes  a  data  set,  which  contains  the  entire  population  density  of  the  League  City 
radar  coverage  area.  This  data  comes  from  the  1990  census  and  was  obtained  from  the 
Socioeconomic  Data  and  Applications  Center  (1995).  The  data  were  averaged  into  a 
2km-by-2km  grid,  thus  containing  4km2  squares  of  data. 

After  these  values  are  read  into  the  program,  it  then  asks  for  a  population  density 
threshold  value.  Using  this  threshold  value,  the  scoring  program  runs  the  same  as  the 
original,  but  excludes  scoring  in  all  areas  that  fall  below  the  threshold  value.  After 
systematically  testing  different  threshold  values,  one  value  should  lead  to  an  optimum 
algorithm  performance  score.  This  optimum  score  can  be  depicted  by  graphing  the 
population  density  threshold  values  versus  the  scoring  values  of  POD,  FAR,  and  CSI. 
This  optimization  technique  should  decrease  the  number  of  false  alarms  that  the 
algorithm  reads  and  ultimately  produce  a  better  reliability  score. 


8)  OPTIMIZING  THE  POSH  PROBABILITY  EQUATION 

Finally,  after  the  variables  of  the  new  WTSM  and  the  values  of  the  optimum 
population  density  threshold  are  determined,  they  can  be  placed  back  into  the  algorithm 
and  run  again  for  the  18  hail  days.  Statistical  values  should  indicate  an  increase  in  the 
CSI,  as  the  WTSM  would  be  optimized  for  the  League  City  radar.  After  ascertaining 
the  new  optimum  CSI  values,  a  new  reliability  graph  can  be  plotted.  This  should  reveal 
an  increased  reliability  in  the  algorithm,  as  it  will  be  more  adapted  to  the  NWS  League 
City  County  warning  area. 

However,  another  way  to  obtain  an  optimized  HD  A  is  to  directly  adjust  the 
parameters  of  the  POSH  probability  equation  (1).  This  equation  is  essentially  a  linear 
function  relating  the  probability  of  severe  hail  to  the  logarithmic  value  of  the  severe  hail 
index  normalized  by  the  warning  threshold  for  that  day.  The  equation  contains  two 
constants,  the  slope  and  intercept  parameter  of  the  linear  relationship.  Either  parameter 
can  be  adjusted.  Hence,  a  straight  forward  adjustment  to  the  HDA  would  be  to  alter 
either  the  intercept  or  the  slope  parameter  in  the  equation  in  order  to  obtain  reliability 
results  closer  to  the  optimum. 
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9)  SENSITIVITY  TO  RADAR  CALIBRATION 

The  League  City  radar  was  recalibrated  after  February  1998.  It  was  found  that 
the  radar  was  under-computing  storm  reflectivities  by  close  to  2  dBZ.  This  can 
sometimes  happen  as  radar  technicians  attempt  to  lengthen  the  life-span  of  the  radar  by 
decreasing  the  power  at  which  the  radar  operates.  However,  doing  this  has  the  ill  effect 
of  having  the  radar  under-compute  storm  reflectivities. 

Since  the  entire  data  set  from  this  study  occurred  before  January  1998,  a 
sensitivity  test  will  be  done  on  a  sub-set  of  the  data  to  ascertain  whether  or  not  this  error 
has  any  effect  on  the  reliability  and  the  performance  of  the  POSH  equation. 
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CHAPTER  IV 
RESULTS 


The  methodology  described  in  the  previous  section  was  used  with  the  data 
available  for  the  League  City  county  warning  area  from  April  1994  through  January 
1998.  These  results  provide  preliminary  information  on  the  NSSL  POSH  algorithm 
over  the  League  City  NWS  office  area  of  interest.  The  first  section  will  briefly 
summerize  the  results  from  a  recent  NSSL  hail  algorithm  study,  which  studied  the 
algorithm’s  performance  over  different  regions  of  the  United  States.  The  second 
section  will  present  the  results  from  this  hail  study,  and  then  compare  these  results  with 
those  from  the  NSSL  study. 

a.  NSSL  hail  study  results 

To  provide  an  independent  test  of  the  SHI  value,  the  initial  WTSM,  and  the 
initial  POSH  probability  function,  NSSL  performed  an  extensive  study  on  their  hail 
algorithm.  This  study  included  storm  data  from  3 1  days  and  7  different  radar  across  the 
United  States  (Witt  et  al.  1998).  Table  5  lists  all  the  radar  locations  used,  along  with  the 
definitions  of  the  separate  regions.  From  the  regions  tested,  the  League  City  forecast 
area  would  most  likely  fall  under  the  Mississippi  River  or  the  Southern  Plains  region. 

The  default  WTSM  was  used  on  these  data  to  run  the  hail  algorithm  and  produce 
performance  statistics.  Table  6  reveals  these  regional  performance  results.  The  SP 
region  appears  to  have  the  highest  scoring  statistics.  This  is  probably  due  to  the  fact 
that  the  default  WTSM  was  developed  using  data  mostly  from  this  area.  Also,  data 
from  this  region  account  for  over  60%  of  the  overall  data  from  this  study. 

The  smallest  variation  throughout  the  regions  appears  to  be  in  the  POD  value, 
having  a  range  size  of  1 1  percentage  points.  This  is  in  direct  contrast  to  the  FAR  and 
CSI  values,  having  ranges  of  29  and  35  percentage  points,  respectively.  One  possibility 
for  this  might  be  that  the  original  WTSM  is  not  highly  representative  of  these  different 
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Table  5.  This  table  shows  the  regions  used  for  the  NSSL  hail  algorithm  study.  NHD 
represents  the  number  of  hail  days  used  in  the  data  set  from  that  region.  NR  represents 
the  number  of  hail  reports  used  in  the  data  set  from  that  region.  The  three-letter  radar 
identifiers  are  included  after  the  radar  location.  From  Witt  et  al.  (1998). 


Radar  Location  (ID) 

Dodge  City,  Kansas  (DDC) 
Norman,  Oklahoma  (OUN) 
Twin  Lakes,  Oklahoma  (TLX) 
Melbourne,  Florida  (MLB) 

St.  Louis,  Missouri  (LSX) 
Memphis,  Tennessee  (NQA) 
Milwaukee,  Wisconsin  (MKX) 
Minneapolis,  Minnesota  (MPX) 
New  York  City,  New  York  (OKX) 


Table  6.  Regional  performance  results  for  the  NSSL  study.  From  Witt  et  al.  (1998). 


Region 

Overall 

POD  (%) 

Overall 

FAR  (%) 

Overall 

CSI  (%) 

Southern  Plains 

82 

54 

41 

Florida 

71 

75 

23 

Mississippi  River 

80 

83 

16 

Northern  United  States 

71 

58 

36 

Average  from  all  regions  in  NSSL  study 

78 

69 

29 

29 


POSH  {%) 


Figure  7.  Same  as  Figure  5,  but  data  comes  from  regional  NSSL  study 
performed  over  3 1  storm  days  and  7  radar  locations.  The  regions  are  labeled  as  follows: 
a)  southern  plains,  b)  Florida,  c)  Mississippi  River,  and  d)  northern  United  States.  Part 
e)  represents  the  average  reliability  of  Figures  9a  through  9d.  Adapted  from  Witt  et  al. 
(1998). 
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regions.  Since  the  WTSM  was  developed  using  radar  data  from  Oklahoma  and  Florida, 
this  is  a  distinct  possibility. 

To  evaluate  the  POSH  parameter,  reliability  diagrams  were  made  to  show  the 
algorithm’s  performance  over  these  regions.  Figures  7a  through  7d  show  the 
algorithm’s  reliability  in  the  separate  regions  tested,  while  Figure  7e  shows  the  average 
reliability  of  the  algorithm  in  all  regions.  As  stated  before,  the  League  City  forecast 
area  would  most  likely  fall  within  the  Southern  Plains  region,  Figure  7a,  or  the 
Mississippi  River  region,  Figure  7c. 

b.  League  City  hail  performance  results 

Following  after  the  methods  used  by  the  NSSL  study,  the  HDA  was  tested  for 
accuracy  over  the  League  City  warning  area.  After  analyzing  the  results  from  the  NSSL 
study,  the  next  step  was  to  tabulate  statistical  data  for  the  League  City  county  warning 
area,  and  then  compare  those  results  with  the  results  from  the  NSSL  test.  For  the  18 
hail  days,  the  algorithm  was  run  using  the  default  WTSM  values  as  determined  by 
NSSL.  The  algorithm  results  from  the  League  City  data  run  are  shown  Tables  7  and  8. 
Table  7  breaks  up  the  data  into  storm  forcing  mechanisms:  seabreeze  and  continental. 
The  performance  statistics  from  the  seabreeze  data  appear  to  be  good,  but  should  not  be 
fully  trusted  due  to  the  fact  that  of  the  120  hail  reports  used,  only  15  of  them  were  due 
to  convection  triggered  by  a  seabreeze  front  (see  Table  1).  The  CSI  results  from  the 
continental  data  were  lower  compared  with  the  overall  averages,  but  this  is  due  to  the 
increased  number  of  false  alarms  within  the  data.  Table  8  breaks  up  the  data  into 
seasons  of  the  year,  where  the  Fall/Winter  months  make  up  33%  of  the  hail  reports. 
Surprisingly,  the  Fall/Winter  months,  where  there  are  not  as  many  hail  reports,  show 
better  algorithm  performance  than  the  Spring/Summer  months. 

When  compared  with  the  regional  results  from  the  NSSL  experiment,  the 
League  City  study  produces  lower  values  for  POD,  but  a  comparable  value  for  the  CSI. 
These  results  show  that  the  default  parameters  for  both  the  WTSM 
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Table  7.  Data  analyzed  from  the  League  City  radar.  H  is  a  hit,  M  is  a  miss,  and  FA  is  a 
false  alarm. 


Seabreeze  Data 

Date 

Ho 

(km) 

WT 

H 

M 

FA 

POD 

FAR 

CSI 

3  Jun  96 

4.50 

138 

7 

1 

4 

88 

36 

58 

2  Jun  96 

4.29 

126 

4 

4 

0 

50 

0 

50 

4  Jun  96 

4.14 

117 

2 

18 

3 

10 

60 

9 

21  May  97 

4.35 

129 

2 

1 

2 

67 

50 

40 

2  Sep  97 

4.30 

126 

1 

1 

2 

50 

67 

25 

Average  from  Seabreeze  Events 

39.02 

40.74 

30.77 

Continental  Data 

Date 

Ho 

(km) 

WT 

H 

M 

FA 

POD 

FAR 

CSI 

21  Jan  98 

3.98 

108 

14 

9 

18 

61 

56 

34 

22  Jan  98 

3.96 

107 

22 

22 

33 

50 

60 

29 

9  Sep  97 

4.60 

144 

11 

12 

13 

48 

54 

31 

3  Nov  95 

4.15 

118 

6 

17 

8 

26 

57 

19 

25  Apr  97 

3.72 

93 

8 

8 

9 

50 

53 

32 

11  Apr  97 

3.70 

92 

12 

18 

7 

40 

37 

32 

28  May  97 

4.10 

115 

4 

4 

7 

50 

64 

27 

2  May  94 

3.96 

107 

15 

5 

5 

75 

25 

60 

5  Apr  94 

4.23 

122 

10 

7 

17 

59 

63 

29 

15  Apr  94 

4.08 

114 

12 

17 

37 

41 

76 

18 

11  May  95 

4.53 

140 

8 

19 

23 

30 

74 

16 

27  Apr  97 

4.42 

133 

2 

23 

5 

8 

71 

7 

5  Apr  96 

3.80 

98 

19 

18 

27 

51 

59 

31 

Average  from  Continental  Events 

44.41 

59.38 

26.93 

Overall  Average  from  League  City  Study 

43.80 

58.05 

27.27 

32 


Table  8.  Same  as  Table  7,  except  data  is  divided  by  time  of  occurrence,  namely  either 


Spring/Summer  or  Fall/Winter. 


Spring/Summer  Data 

Date 

Ho 

(km) 

WT 

H 

M 

FA 

POD 

FAR 

CSI 

3  Jun  96 

4.50 

138 

7 

1 

4 

88 

36 

58 

2  Jun  96 

4.29 

126 

4 

4 

0 

50 

0 

50 

4  Jun  96 

4.14 

117 

2 

18 

3 

10 

60 

9 

21  May  97 

4.35 

129 

2 

1 

2 

67 

50 

40 

25  Apr  97 

3.72 

93 

8 

8 

9 

50 

53 

32 

11  Apr  97 

3.70 

92 

12 

18 

7 

40 

37 

32 

28  May  97 

4.10 

115 

4 

4 

7 

50 

64 

27 

2  May  94 

3.96 

107 

15 

5 

5 

75 

25 

60 

5  Apr  94 

4.23 

122 

10 

7 

17 

59 

63 

29 

15  Apr  94 

4.08 

114 

12 

17 

37 

41 

76 

18 

11  May  95 

4.53 

140 

8 

19 

30 

74 

16 

27  Apr  97 

4.42 

133 

2 

23 

5 

8 

71 

7 

5  Apr  96 

3.80 

98 

19 

18 

27 

51 

59 

31 

Average  from  Spring/Summer  Events 

42.34 

58.17 

26.65 

Fall/Winter  Data 

Date 

Ho 

(km) 

WT 

H 

M 

FA 

POD 

FAR 

CSI 

2  Sep  97 

4.30 

126 

1 

1 

2 

50 

67 

25 

21  Jan  98 

3.98 

108 

14 

9 

18 

61 

56 

34 

22  Jan  98 

3.96 

107 

22 

22 

33 

50 

60 

29 

9  Sep  97 

4.60 

144 

11 

12 

13 

48 

54 

31 

3  Nov  95 

4.15 

118 

6 

17 

8 

26 

57 

19 

Average : 

'rom  Fall/Winter  Events 

46.95 

57.81 

28.57 

33 


and  the  POSH  probability  function  create  CSI  statistics  equivalent  to  those  of  the  NSSL 
study,  and  thereby  could  be  considered  accurate  for  the  League  City  warning  area. 
However,  another  way  to  test  the  accuracy  of  the  HDA  is  to  determine  whether  or  not 
the  WTSM  remains  highly  correlated  with  the  melting  level.  If  so,  then  optimizing  the 
WTSM  parameters  will  be  the  next  objective. 

1)  WTSM  GRAPHS 

For  each  day  listed  on  Table  7,  an  optimum  warning  threshold  was  determined 
using  the  scoring  programs.  This  data  was  plotted  against  the  melting  level  of  the  day. 
Since  the  performance  results  were  divided  into  different  classifications,  so  too  were  the 
data  for  the  WTSM  graphs.  This  is  shown  on  Figures  8a  through  8h.  The  seabreeze 
data  appears  to  show  a  well-correlated  WTSM  line.  However,  these  results  are  suspect 
due  to  the  lack  of  data.  As  well,  the  Spring/Summer  data  appears  to  be  more  correlated 
than  does  the  Fall/Winter  data. 

The  Spring/Summer  data  were  further  separated  by  continental  and  seabreeze 
convective  forcing.  This  proved  to  be  useful  as  the  continental-Spring/Summer  data 
were  highly  correlated.  This  was  also  useful  because  half  of  the  data  set  was  contained 
in  this  subset.  The  seabreeze-Spring/Summer  data  was  also  highly  correlated,  but 
again,  the  lack  of  data  lessens  the  credibility  of  the  results.  Lastly,  separating  the 
continental-Fall/Winter  data  showed  relatively  no  correlation. 

After  analyzing  the  results  from  these  graphs,  the  next  step  was  to  select  a  new 
WTSM  to  use  in  the  final  analysis  of  the  NSSL  hail  algorithm.  The  line  with  the 
highest  correlation  could  be  used,  but  this  appears  in  the  seabreeze  data  (Figure  8b), 
where  data  points  are  few.  This  eliminates  the  usage  of  the  seabreeze  and  seabreeze- 
Spring/Summer  (Figure  8f)  plots.  Another  consideration  has  to  do  with  the  limilations 
of  the  WATADS  software.  When  inputting  new  parameters  into  the  hail  algorithm, 
WATADS  only  allows  a  window  of  -500  to  +500  for  the  y-intercept  value.  This 
disallows  another  highly  correlated  plot,  the  continental-Spring/Summer  plot  (Figure 
8e),  with  a  correlation  of  .83.  Therefore,  the  slope  and  y-intercept  values  from  the 


Optimum  Warning  Threshold  (J/ms)  Optimum  Warning  Threshold  (J/ms) 
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Continental  Data 


WT  =  118.3Ho- 373.43 
R2  =  .59 


Figure  8a.  WTSM  plot  for  all  continental-forced  hail  days.  The  black 
line  represents  the  “best  fit”  linear  correlation  of  the  data,  which  is  shown 
numerically  to  the  left  of  the  plot.  The  R2  value  represents  the  correlation  value 
of  this  line. 


Seabreeze  Data 
WT  =  52.38H0- 131.10 
R2  =  .79 


4.1  4.2  4.3  4.4  4.5  4.6 

Melting  Layer  (H^  in  km 

Figure  8b.  Same  as  Fig.  8a,  but  for  all  seabreeze-forced  hail  days. 
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Snring/Summer  Data 
WT  =  130.43Ho-  435.0 
R2  =  .63 


3.6  3.8  4  4.2  4.4  4.6 

Melting  Layer  (HJ  In  km 

Figure  8c.  Same  as  Fig.  8a,  but  for  all  Spring/Summer  hail  days. 


Fall/Winter  Data 
WT  =  -28.65Ho  +  234.27 
R2  =  .26 


Melting  Layer  (Ho)  in  km 

Figure  8d.  Same  as  Fig.  8a,  but  for  all  Fall/Winter  hail  days. 


Optimum  Warning  Threshold  (J/ms)  Optimum  Warning  Threshold  (Jtms) 
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Melting  Layer  (H  )  in  km 

Figure  8e.  Same  as  Fig.  8a,  but  for  all  continental  Spring/Summer  hail  days. 


Mettlng  Layer  (Ho)  In  km 


Figure  8f.  Same  as  Fig  8a,  but  for  all  seabreeze-Spring/Summer  hail  days. 


Figure  8h.  Same  as  Fig  8a,  but  for  all  18  hail  days. 


38 


Spring/Summer  plot,  with  a  correlation  of  .73,  will  be  used  to  analyze  the  hail  algorithm 
for  the  League  City  warning  area.  These  WT  values  are  shown  on  Figure  8c. 

2)  POSH  RELIABILITY 

Another  way  to  test  the  accuracy  of  the  hail  detection  algorithm  over  the  area  of 
study  is  to  plot  the  reliability  of  the  POSH  equation.  To  accomplish  this,  the  specific 
POSH  values  were  tabulated  into  two  categories;  predicted  POSH  values  from  the 
algorithm  output  files  and  actual  POSH  values  from  the  hail-truth  files.  These  data 
were  then  compared  with  each  other  and  placed  onto  a  reliability  graph,  which  is  shown 
on  Figure  9.  From  this  graph,  one  can  see  an  over-forecasting  bias  by  the  algorithm  for 
every  POSH  value.  Also,  the  reliability  seems  to  worsen  after  80%.  However,  this  is 
due  to  the  lack  of  observations  at  both  90%  and  100%.  A  simple  cumulative 
distribution  graph  (Figure  10)  shows  this  to  be  true.  The  forecasted  values  of  POSH 
from  10  to  80  account  for  98%  of  the  data.  Therefore,  the  results  shown  for  the  last  two 
POSH  values  are  based  on  a  data  sample  too  small  to  be  counted  as  significant. 

Most  of  the  data  points  appear  to  be  progressing  at  a  linear  rate,  suggesting  that, 
as  well  as  testing  a  new  WTSM,  it  also  may  be  useful  to  parameterize  the  offset 
parameter  of  the  POSH  probability  function  using  the  default  WTSM. 

3)  POPULATION  DENSITY  CONSIDERATIONS 

Before  the  hail  algorithm  was  run  using  the  new  parameters,  the  issue  of  a 
population  density  bias  in  the  data  was  addressed.  Using  the  findings  and  threshold 
values  from  Wyatt  and  Witt  (1998)  concerning  population  density  bias,  a  special 
program  was  run  that  filters  out  data  below  certain  population  density  thresholds.  The 
thresholds,  in  units  of  persons  per  4km2,  used  for  this  study  are  listed  as  follows:  0,  100, 
250,  500,  750  and  1000.  Figure  11  shows  the  results  from  running  the  hail  algorithm 
with  the  population  restrictions.  The  results  from  this  exercise  show  the  highest  CSI 
and  POD  values  appear  when  there  are  no  population  restrictions  on  the  data.  Wyatt 
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POSH (%) 

Figure  9.  This  is  the  reliability  graph  for  the  League  City  county  warning  area 
using  only  the  default  algorithm  values.  The  solid  black  line  represents  the  optimum 
reliability.  The  circles  represent  the  actual  reliabilities  of  each  POSH  value. 


0  1  0  20  30  40  50  60  70  80  90  100 


POSH  (%) 


Figure  10.  This  is  a  graph  showing  the  cumulative  frequency  distributions  of 
both  the  observed  and  forecasted  POSH  values.  The  default  algorithm  parameters  were 
used  to  calculate  this  data. 
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Population  Density  Threshold  (persons  per  4km2) 


Figure  11.  This  figure  shows  the  statistical  scoring  values  plotted  against  certain 
population  density  thresholds. 


Population  Density  Threshold  Values  (persons  per  4km*) 


Figure  12.  This  figure  shows  the  cumulative  frequency  distribution  of  the 
observed  hail  occurrences  based  on  population  density  threshold  values  versus  the 


population  density  distributions  over  the  League  City  county  warning  area. 
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and  Witt  (1998)  came  across  this  in  their  study  as  well.  They  examined  two  radar 
locations:  Melbourne,  Florida  and  Fort  Worth,  Texas.  In  the  Florida  case  they  saw 
maximum  performance  statistics  at  the  1000  persons  per  4km2  point.  However,  the  Fort 
Worth  case  showed  the  best  hail  performance  statistics  at  0  persons  per  4km2. 

The  reason  for  this  behavior  is  most  likely  in  the  data-gathering  process  itself. 
Wyatt  and  Witt  (1998)  claimed  that  the  reason  the  Fort  Worth  data  behaved  as  it  did 
was  due  to  the  fact  that  most  of  the  hail  reports  were  from  rural  areas  (less  than  100 
persons  per  4km2).  A  cumulative  frequency  distribution  of  hail  reports  sorted  by 
population  density  over  the  League  City  county  warning  area  (Fig.  12)  illustrates  this 
for  the  current  study  to  some  point.  The  distribution  of  hail  reports  can  be  compared  to 
the  fractional  area  of  the  county  warning  area  covered  by  each  increment  in  population 
density  threshold  (the  dashed  curve  in  Fig.  12).  Close  to  85%  of  the  total  area  within 
this  county  warning  area  have  population  densities  <100  persons  per  4km2.  However, 
only  approximately  55%  of  the  observed  hail  occurrences  for  this  study  occurred  within 
this  population  threshold.  Other  thresholds  on  the  graph  point  to  the  same  conclusion 
—  that  there  is  a  likelihood  that  the  gathering  of  hail  reports  solely  from  Storm  Data 
will  tend  to  bias  the  hail-truth  file  towards  more  largely  populated  areas.  While 
expected,  this  illustrates  the  difficulty  in  obtaining  validation  information  that  is 
representative  of  the  natural  variability  of  hail  events.  With  most  of  the  hail  reports  for 
this  study  coming  from  rural  areas,  testing  for  an  optimum  population  density  threshold 
proved  unfruitful  in  lessening  the  FAR  for  a  more  reliable  score. 

4)  OPTIMIZED  WTSM  RESULTS 

The  statistical  results  from  running  the  hail  algorithm  with  the  new  WTSM  are 
presented  on  Tables  9  and  10.  There  is  an  increase  in  the  CSI  and  POD  values,  and  a 
decrease  in  the  values  for  FAR.  This  statistical  increase  may  indicate  that  the  default 
WTSM  parameters  were  not  optimal  over  the  League  City  data  set.  Using  the  new 
WTSM,  a  new  reliability  graph  was  plotted  and  is  shown  on  Figure  13.  This  shows  a 
good  improvement  of  the  algorithm  for  the  0%  to  40%  POSH  values.  From  40%  to 
60%  the  new  reliability  is  better  than  the  old,  but  there  is  still  a  distinct  over-forecasting 
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Table  9.  Same  as  Table  7,  except  this  table  shows  the  statistical  results  from  running 


the  hail  algorithm  using  an  optimized  WTSM  line. 


Seabreeze  Data 

Date 

Ho 

(km) 

WT 

H 

M 

FA 

POD 

FAR 

CSI 

3  Jun  96 

4.50  . 

150 

7 

1 

4 

88 

36 

58 

2  Jun  96 

4.29 

123 

4 

4 

0 

50 

0 

50 

4  Jun  96 

4.14 

103 

4 

16 

9 

20 

69 

14 

21  May  97 

4.35 

131 

2 

1 

0 

67 

0 

67 

2  Sep  97 

4.30 

124 

1 

1 

0 

50 

0 

50 

Average  from  Seabreeze  Events 

43.90 

41.94 

33.33 

Continental  Data 

Date 

Ho 

(km) 

WT 

H 

M 

FA 

POD 

FAR 

CSI 

21  Jan  98 

3.98 

82 

15 

8 

27 

65 

64 

30 

22  Jan  98 

3.96 

80 

23 

22 

39 

51 

63 

27 

9  Sep  97 

4.60 

163 

10 

13 

10 

44 

50 

30 

3  Nov  95 

4.15 

105 

8 

15 

\  14 

35 

64 

22 

25  Apr  97 

3.72 

49 

13 

3 

1  12 

81 

48 

46 

11  Apr  97 

3.70 

46 

19 

9 

12 

68 

39 

48 

28  May  97 

4.10 

98 

4 

4 

7 

50 

64 

27 

2  May  94 

3.96 

80 

17 

3 

7 

85 

29 

63 

5  Apr  94 

4.23 

115 

10 

8 

18 

56 

64 

28 

15  Apr  94 

4.08 

95 

20 

9 

44 

69 

69 

27 

11  May  95 

4.53 

154 

6 

20 

20 

23 

77 

13 

27  Apr  97 

4.42 

140 

2 

23 

5 

8 

71 

7 

5  Apr  96 

3.80 

59 

24 

9 

33 

73 

58 

36 

Average  from  Continental  Events 

53.94 

59.19 

30.27 

Overall  Average  from  League  City  Study 

52.79 

58.00 

30.53 
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Table  10.  Same  as  Table  8,  except  this  table  shows  the  statistical  results  from  running 
the  hail  algorithm  using  an  optimized  WTSM  line. 


Fall/Winter  Data 

Date 

Ho 

(km) 

WT 

H 

M 

FA 

POD 

FAR 

CSI 

2  Sep  97 

4.30 

90 

3 

0 

0 

100 

0 

100 

21  Jan  98 

3.98 

110 

13 

9 

13 

59 

50 

37 

22  Jan  98 

3.96 

160 

22 

22 

21 

43 

37 

35 

9  Sep  97 

4.60 

120 

14 

9 

17 

61 

55 

35 

3  Nov  95 

4.15 

90 

8 

15 

14 

35 

64 

22 

Average  from  Fall/Winter  Events 

52.18 

52.00 

33.33 

44 


Figure  13.  Same  as  Figure  9,  except  this  figure  includes  the  results  from 
running  the  hail  algorithm  using  the  optimized  WT  values  for  the  League  City  warning 
area. 
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Figure  14.  Same  as  Figure  10,  except  this  figure  shows  the  statistical  results 
from  running  the  hail  algorithm  using  an  optimized  WTSM  line. 
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bias.  From  60%  to  100%  the  new  values  are  worse  than  the  old,  but  this  could  be  due  to 
the  limited  number  of  POSH  values  greater  than  60%. 

The  forecasted  values  of  POSH  from  10  to  60  account  for  80%  of  the  data,  as 
shown  by  the  cumulative  distribution  graph  on  Figure  14.  Still,  the  bulk  of  the  data  is 
found  between  POSH  values  of  10%  to  60%.  It  is  not  clear  if  the  decrease  in 
performance  at  the  high  end  of  the  POSH  values  is  associated  with  a  lack  of  high 
quality  validation  data  or  an  inherent  limitation  of  the  existing  algorithm.  Since  testing 
the  WTSM  with  the  reliability  graph  is  met  with  marginal  success,  the  next  step  is  to 
adjust  the  POSH  probability  function  using  the  default  WTSM  values.  This  should 
produce  similar  results  as  optimizing  the  WTSM. 


5)  OPTIMIZING  THE  POSH  PROBABILITY  EQUATION 


Visual  inspection  of  the  reliability  graph  (Figure  9)  suggests  that  over  most  of 
the  POSH  range,  the  default  algorithm  over  warned  by  about  20%.  Hence,  a  straight¬ 
forward  adjustment  to  the  HDA  would  be  to  reduce  the  intercept  parameter  in  the  POSH 
equation  by  20  so  that: 


POSH'  =29  In 


SHP 
WT , 


+  30 


(7) 


Where  POSH'  is  the  optimized  value  of  POSH  for  this  data  set.  All  18  hail  days  were 
run  through  the  hail  algorithm  again  using  the  new  POSH  equation. 

The  results  are  shown  on  Figure  15.  This  shows  reliabilities  very  near  optimum 
from  10%  to  60%.  However,  from  60%  up  to  100%  the  results  seem  to  be  very 
convoluted.  As  before,  plotting  a  cumulative  distribution  graph  shows  that  the 
reliability’s  behavior  from  60%  to  100%  is  due  to  poor  sampling  at  those  levels.  Figure 
16  illustrates  this  very  clearly.  The  percentage  of  forecasted  POSH  data  contained 
between  10%  and  60%  is  97%.  Therefore,  due  to  low  sampling  of  POSH  values  greater 
than  60%,  it  is  impossible  to  gather  any  reasonable  interpretation  from  the  POSH 
reliability  results  over  these  values.  However,  Tables  1 1  and  12  show  that  when  the 


Observed  Relative  Frequency  (%) 
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POSH  (%) 


-  ♦  -  POSH  Reliability  (default  values) 

- Optimum  Score 

--♦  -POSH  Reliability  (new  WT  values) 

— ■ —  POSH  Reliability  (new  POSH  values) 


Figure  15.  Same  as  Figure  13,  except  this  figure  includes  the  results  from 
altering  the  POSH  probability  equation. 
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Figure  16.  Same  as  Figure  10,  except  this  figure  shows  the  statistical  results 
from  running  the  hail  algorithm  after  altering  the  POSH  probability  equation. 


Table  11.  Same  as  Table  9,  except  this  table  shows  the  statistical  results  from  running 


the  hail  algorithm  using  an  optimized  POSH  equation. 


Seabreeze  Data 

Date 

Ho 

(km) 

WT 

H 

M 

FA 

POD 

FAR 

CSI 

3  Jun  96 

4.50 

138 

7 

1 

3 

88 

30 

64 

2  Jun  96 

4.29 

126 

4 

4 

0 

50 

0 

50 

4  Jun  96 

4.14 

117 

3 

17 

7 

15 

70 

11 

21  May  97 

4.35 

129 

4 

0 

1 

100 

20 

80 

2  Sep  97 

4.30 

126 

1 

1 

1 

50 

50 

33 

Average  from  Seabreeze  Events 

45.24 

38.71 

35.19 

Continental  Data 

Date 

Ho 

(km) 

WT 

H 

M 

FA 

POD 

FAR 

CSI 

21  Jan  98 

3.98 

108 

13 

9 

13 

59 

50 

37 

22  Jan  98 

3.96 

107 

19 

25 

11 

43 

37 

35 

9  Sep  97 

4.60 

144 

11 

12 

13 

48 

54 

31 

3  Nov  95 

4.15 

118 

7 

16 

12 

30 

63 

20 

25  Apr  97 

3.72 

93 

6 

10 

4 

38 

40 

30 

11  Apr  97 

3.70 

92 

7 

23 

4 

23 

36 

21 

28  May  97 

4.10 

1  f  5 

4 

4 

1 

50 

20 

44 

2  May  94 

3.96 

107 

19 

1 

8 

95 

30 

68 

5  Apr  94 

4.23 

122 

10 

7 

7 

59 

41 

42 

15  Apr  94 

4.08 

114 

13 

16 

36 

45 

74 

20 

1 1  May  95 

4.53 

140 

8 

19 

23 

30 

74 

16 

27  Apr  97 

4.42 

133 

2 

23 

4 

8 

67 

7 

5  Apr  96 

3.80 

98 

25 

8 

34 

76 

58 

37 

Average  1 

rom  Continental  Events 

45.43 

54.14 

29.75 

Overall  Average  from  League  City  Study 

45.40 

52.75 

30.13 

49 


Table  12.  Same  as  Table  10,  except  this  table  shows  the  statistical  results  from  running 


the  hail  algorithm  using  an  optimized  POSH  equation. 


Spring/Summer  Data 

Date 

Ho 

(km) 

WT 

H 

M 

FA 

POD 

FAR 

CSI 

3  Jun  96 

4.50 

138 

7 

1 

3 

88 

30 

64 

2  Jun  96 

4.29 

126 

4 

4 

0 

50 

0 

50 

4  Jun  96 

4.14 

117 

3 

17 

7 

15 

70 

11 

21  May  97 

4.35 

129 

4 

0 

1 

100 

20 

80 

25  Apr  97 

3.72 

93 

6 

10 

4 

38 

40 

30 

1 1  Apr  97 

3.70 

92 

7 

23 

4 

23 

36 

21 

28  May  97 

4.10 

115 

4 

4 

1 

50 

20 

44 

2  May  94 

3.96 

107 

19 

1 

8 

95 

30 

68 

5  Apr  94 

4.23 

122 

10 

7 

7 

59 

41 

42 

15  Apr  94 

4.08 

114 

13 

16 

36 

45 

74 

20 

11  May  95 

4.53 

140 

8 

19 

23 

30 

74 

16 

27  Apr  97 

4.42 

133 

2 

23 

4 

8 

67 

7 

5  Apr  96 

3.80 

98 

25 

8 

34 

76 

58 

37 

Average  from  Spring/Summer  Events 

45.71 

54.10 

29.71 

Fall/Winter  Data 

Date 

Ho 

(km) 

WT 

H 

M 

FA 

POD 

FAR 

CSI 

2  Sep  97 

4.30 

126 

1 

1 

1 

50 

50 

33 

21  Jan  98 

3.98 

108 

13 

9 

13 

59 

50 

37 

22  Jan  98 

3.96 

107 

19 

25 

11 

43 

37 

35 

9  Sep  97 

4.60 

144 

11 

12 

13 

48 

54 

31 

3  Nov  95 

4.15 

118 

7 

16 

12 

30 

63 

20 

Average : 

rom  Fall/Winter  Events 

44.74 

49.50 

31.10 

50 


scoring  algorithm  was  run  with  the  new  POSH  output,  it  produced  similar  statistical 
results  as  the  ones  from  the  new  WTSM. 

6)  RADAR  CALIBRATION  SENSITIVITY  RESULTS 

All  reflectivities  from  a  chosen  subset  of  the  data  were  increased  by  2  dBZ  and 
ran  again  through  the  scoring  algorithm.  Performance  statistics  were  computed  (shown 
on  Table  13),  as  well  as  the  reliability  of  the  POSH  algorithm  (shown  on  Figure  17). 

When  compared  with  the  statistics  from  the  default  algorithm  run,  the  POD 
values  looked  very  similar.  However,  in  the  dBZ-adjusted  run  the  FAR  values  were 
close  to  14%  higher  than  the  default  values.  This  resulted  in  the  dBZ-adjusted  run 
having  an  approximate  10%  decrease  in  CSI  when  compared  with  the  default  run.  The 
reliability  graph  shows  the  dBZ-adjusted  values  with  more  optimal  reliability  values 
from  POSH  =  10%-30%.  However,  with  the  exception  of  POSH  =  60%,  the  adjusted 
values  become  less  reliable  than  the  default  run  after  POSH  =  40%. 

Looking  at  the  distribution  of  forecasted  POSH  values  between  the  default  run 
and  the  dBZ-adjusted  run  illustrates  the  same  results  (Figure  18).  In  the  lower  values  of 
POSH,  the  dBZ-adjusted  POSH  values  are  lower  than  the  default  values.  This  would 
account  for  the  higher  reliability  values  in  Figure  17.  However,  as  POSH  increases  the 
dBZ-adjusted  values  become  greater  than  the  default  values.  This  shows  that  in  the 
dBZ-adjusted  run  there  is  greater  over- forecasting  of  the  higher  values  of  POSH. 
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Table  13.  This  table  shows  the  statistical  performance  of  the  HDA  when  2  dBZ  were 
added  to  all  reflectivity  values.  The  total  default  values  listed  below  come  only  from 
the  5  days  tested. 

Date  POD  (%)  FAR  (%)  CSI  (%) 


15  Apr  94 

38 

76 

17 

03  Jun  96 

88 

30 

64 

04  Jun  96 

10 

60 

09 

25  Apr  97 

56 

67 

26 

21  Jan  98 

67 

63 

32 

22  Jan  98 

49 

55 

31 

Total  Values 

45 

67 

26 

Total  Default 
Values 

44 

58 

27 
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Figure  17.  This  figure  shows  the  change  in  the  reliability  of  a  subset  of  data 
used  to  test  the  radar  sensitivity  to  a  2  dBZ  reflectivity  adjustment.  Default  run  values 
are  shown  by  the  circles.  The  POSH  values  of  the  dBZ-adjustment  run  are  shown  by 
the  open  diamonds. 


Frequency  of  Forecasted  POSH  (%) 
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Figure  18.  This  figure  shows  the  frequency  of  forecasted  POSH  values  when 
the  algorithm  was  run  using  the  2  dBZ  adjustment.  The  default  run  values  are  shown  in 
black,  while  the  dbz-adjusted  values  are  shown  in  gray. 
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CHAPTER  V 
CONCLUSIONS 


The  current  version  of  WATADS  contains  several  algorithms  for  detecting 
severe  thunderstorms.  They  were  built  and  installed  in  NWS  offices  with  the  intent  that 
testing  and  optimization  would  be  done  by  local  offices.  This  becomes  very  important 
for  empirically  derived  statistical  algorithms  that  are  tested  in  climates  different  from 
that  for  which  they  were  developed.  Unfortunately,  these  studies  can  become  laborious 
and  time-consuming  for  those  examining  the  data.  A  simple  method  to  optimize  the 
hail  algorithm  over  various  regions  of  the  U.S.  would  save  NWS  offices  time  and  allow 
them  to  implement  these  changes  in  a  quick  and  timely  manner. 

This  study  examined  the  performance  of  the  NSSL  hail  algorithm  for  18 
different  storm  systems  over  the  Gulf  coast  region  in  eastern  Texas.  This  study  ran  the 
hail  algorithm  with  all  of  the  default  values  and  then  compared  them  with  results  from  a 
NSSL  regional  hail  study.  It  then  presented  adjustments  to  the  POSH  output  of  the 
current  hail  detection  algorithm.  These  adjustments  include  optimizing  the  WTSM  line, 
population  density,  and  the  POSH  probability  function.  Using  the  default  parameters, 
the  hail  detection  algorithm  over  warned  by  about  20%  over  most  of  the  range  of  values 
for  the  probability  of  severe  hail. 

When  run  with  the  default  values,  the  algorithm’s  POD  score  for  the  League 
City  area  is  lower  than  any  region  in  the  NSSL  study.  This  would  indicate  that  for  this 
study  there  are  a  copious  amount  of  algorithm  misses  for  severe  hail.  However,  this 
could  be  due  to  severe  hail  occurrences  that  failed  to  produce  severe  hail  values  for  the 
entire  20-minute  time-window.  Still,  both  the  FAR  and  CSI  scores  were  comparable  to 
the  overall  regional  scores  from  the  NSSL  study.  This  could  indicate  initially  that  the 
hail  algorithm  runs  optimally  using  the  default  values  over  the  League  City  warning 
area. 

Testing  to  see  if  the  WT  was  correlated  with  the  melting  layer  indicated  the 
continental-Spring/Summer  and  the  Spring/Summer  data  sets  had  good  relationships 
between  the  optimal  WT  and  the  melting  layer.  The  WTSM  values  chosen  as  optimal 
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were  approximately  two  times  larger  than  the  default  values.  These  values  produced 
better  statistical  values  than  the  default  algorithm  run,  and  could  therefore  be  considered 
optimal  for  that  data  set,  and  possibly  the  region. 

As  well,  optimizing  the  algorithm  by  changing  the  WTSM  to  provide  the  highest 
CSI  for  this  data  set  improved  the  reliability  of  the  POSH  function  for  POSH  values  less 
than  60%.  For  reasons  that  may  be  related  to  the  lack  of  sufficient  observations  for  high 
POSH  values,  the  performance  of  the  optimized  algorithm  decreased  relative  to  the 
default  algorithm  for  POSH  values  greater  than  60%. 

Using  a  population  density  program  to  filter  out  hail  data  below  a  certain 
population  threshold  showed  no  improvement  in  the  statistical  scores.  The  highest 
overall  POD,  FAR,  and  CSI  scores  appeared  when  the  population  density  thresholds 
were  set  at  zero.  This  was  predicted  to  occur  when  most  of  the  hail  data  comes  from 
rural  areas  (less  than  100  persons  per  4km2).  This  was  shown  to  be  true  as  the 
percentage  of  convective  activity  that  occurred  over  the  most  populated  region  (1000 
persons  per  4km2)  only  included  approximately  5%  of  what  occurred  over  the  entire 
data  set.  Therefore,  it  was  not  necessary  to  use  any  population  density  threshold  for  the 
data  set  of  this  study. 

Since  the  over  warning  of  the  default  algorithm  was  nearly  constant,  better 
reliability  results  were  obtained  by  optimizing  the  algorithm  directly  through  the  POSH 
equation.  The  offset  value  of  the  POSH  probability  function  was  adjusted  from  +50  to 
+30.  This  alteration  showed  results  very  close  to  optimum  on  the  reliability  graph  from 
POSH  values  10%  to  60%.  Values  greater  than  60%  showed  reliability  values  worse 
than  the  default  values.  However,  the  POSH  values  greater  than  60%  accounted  for 
only  3%  of  the  entire  data  set.  The  lack  of  observations  associated  with  high  POSH 
values  limits  the  proof  of  improvement  in  algorithm  performance  to  POSH  values  less 
than  60%,  and  therefore,  can  be  considered  inconclusive.  Statistical  performance  from 
the  optimized  POSH  algorithm  run  showed  results  almost  the  same  as  the  results  from 
the  optimized  WTSM. 

Two  methods  were  used  to  optimize  the  POSH  over  the  League  City  warning 
area.  Optimizing  the  WTSM  showed  improvement  of  the  statistical  performance  of  the 
algorithm,  but  showed  more  ambiguous  results  with  the  reliability  graph.  Optimizing 
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the  reliability  of  the  POSH  equation  showed  a  more  optimal  graphical  improvement, 
and  showed  the  same  statistical  improvement  to  the  HDA  as  did  the  newly  optimized 
WTSM.  This  result  is  to  be  expected  as  multiplying  the  WT  by  a  factor  of  two 
mathematically  gives  close  to  the  same  result  as  subtracting  20  from  the  POSH  offset 
value  (see  Appendix  B). 

Even  though  optimizing  both  the  WTSM  and  POSH  showed  improvements  to 
the  algorithm  scores,  it  more  importantly  shows  that  a  detailed  WTSM  study  for  a 
particular  area  may  not  be  necessary,  as  optimizing  the  POSH  equation  through  the 
reliability  graph  gives  similar  results.  Simply  optimizing  the  POSH  equation  could  be  a 
great  time-saver  as  intensive  warning  threshold  studies  would  require  copious  amounts 
of  time  in  data-gathering  and  analysis.  This  could  allow  NWS  offices  to  optimize  their 
HDA  more  rapidly,  thereby  providing  more  accurate  warnings  to  their  area  of 
responsibility. 

The  sensitivity  to  changes  in  the  reflectivity  values  was  also  tested  in  this  study. 
The  results  from  the  dBZ  calculation  error  show  that  the  performance  of  the  HDA  is 
susceptible  to  small  variations  in  the  interpretation  of  the  reflectivity  values  by  the 
radar.  This  finding  has  profound  implications  as  it  could  lessen  the  validity  of  earlier 
hail  algorithm  studies  if  the  radars  used  were  not  calibrated  properly.  Before  any 
parameter  adjustment  to  the  hail  algorithm  is  done,  measures  should  be  taken  by  each 
NWS  office  to  ensure  calibration  is  done  to  their  radar. 

With  severe  hail  being  a  significant  hazardous  weather  threat,  it  is  important  that 
a  hail  detection  algorithm  provide  useful  guidance  to  forecasters.  The  ability  for 
weather  software  to  include  adjustable  parameters  in  its  algorithms  greatly  increases  its 
usefulness  to  field  operators.  However,  computer  software  is  only  as  good  as  the 
hardware  it  runs  on.  To  provide  the  most  accurate  adjustments  to  the  hail  algorithm 
software,  the  responsibility  falls  upon  individual  NWS  offices  to  ensure  their  radar 
equipment  is  in  proper  working  condition.  The  adjustments  made  to  the  POSH 
probability  function  in  this  study  should  improve  the  over-forecasting  perception  of  the 
HDA,  and  hopefully  lead  to  more  accurate,  effective  hail  forecasts  over  the  Texas  Gulf 
coast  in  the  future. 
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APPENDIX  A 

HAILSTORM  INDICATORS 


From  the  findings  of  Lemon  (1978),  several  hail  indicators  were  determined  to 
be  the  most  useful  in  representing  the  storm  structure  and  the  conditions  necessary  for 
hailstorm  development  (Smart  and  Alberty  1985).  They  are  listed  on  Table  Al.  The 
hail  algorithm  used  information  from  the  Storm  Centroid  and  Tracking  Algorithms  to 
test  for  the  presence  of  these  indicators  (Smart  and  Alberty  1985).  Figure  Al  shows  an 
illustration  of  a  modeled  hailstorm  by  the  Storm  Centroid  Algorithm.  After  testing  was 
completed,  the  storm  was  given  one  of  the  following  four  hail  (of  any  size)  labels: 
Positive,  Probable,  Negative  or  Unknown  (insufficient  data  available  to  make  a 
decision). 
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Table  Al.  Hail  Indicators  and  their  Assigned  Weight  Values.  From  Smart  and 
Alberty  (1985) 
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Figure  Al.  Illustration  of  a  hailstorm  as  modeled  by  the  NEXRAD  storm 
identification  algorithms.  The  numbers  indicate  five  of  the  seven  hail  indicators  listed 
on  Table  1.  Adapted  from  Smart  and  Alberty  (1985). 
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APPENDIX  B 

DEFINITIONS  OF  RELAVENT  ACRONYMS 


CSI 

Critical  Success  Index 

dBZ 

Logarithmic  unit  of  (Z) 

E 

Hailfall  Kinetic  Energy 

FAR 

False  Alarm  Rate 

HCAA 

Hail  Core  Aloft  Algorithm 

HDA 

Hail  Detection  Algorithm 

MEHS 

Maximum  Expected  Hail  Size 

NCDC 

National  Climatic  Data  Center 

NSSL 

National  Severe  Storms  Laboratory 

NWS 

National  Weather  Center 

ORF 

Observed  Relative  Frequency 

POD 

Probability  of  Detection 

POH 

Probability  of  Hail 

POSH 

Probability  of  Severe  Hail 

SCIT 

Storm  Cell  Identification  and  Tracking  Algorithm 

SHI 

Severe  Hail  Index 

ULDA 

Upper  Level  Divergence  Algorithm 

VIL 

Vertically  Integrated  Liquid  Water  Content 

WATADS 

WSR-88D  Algorithm  Testing  and  Display  System 

WSR-88D 

Weather  Surveillance  Radar-1988  Doppler 

W(Z) 

Reflectivity  Weighting  Function 

WT 

Warning  Threshold 

Wt(H) 

Temperature-based  Weighting  Function 

WTSM 

Warning  Threshold  Selection  Model 
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APPENDIX  C 
dBZ  EXPLANATION 

Simply  put,  dBZ  is  the  logarithmic  unit  of  the  radar  reflectivity  factor  (Z).  The 
radar  reflectivity  factor  is  defined  as  the  sum  of  the  particle  diameters  raised  to  the  sixth 
power  per  unit  volume  space  sampled  by  the  radar  pulse  (Battan  1973). 
Mathematically,  Z  is  given  by: 

z=  Id6 

volume 

where  D  is  the  diameter  of  the  particle.  The  radar  reflectivity  factor  is  calculated  for 
the  return  power  measured  by  the  radar  receiver.  These  values  can  have  a  wide  range 
depending  upon  the  size  of  the  particle  observed  by  the  radar.  They  can  range  from 
small  particles,  such  as  fog,  which  has  a  sample  volume  size  of  approximately  0.001 
mm  m  ,  all  the  way  up  to  very  large  particles,  such  as  hail,  which  can  have  a  volume 
size  of  36,000,000  mm6  nT3  for  softball-sized  hail. 

In  order  to  account  for  these  enormous  variations  in  volume  size,  it  is  convenient 
to  compress  these  values  using  a  logarithmic  scale.  This  leads  to  following  equation,  in 
which  dBZ  is  the  logarithmic  reflectivity  factor. 


dBZ  -  101og10 


Z 

1mm6  /  m3 
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APPENDIX  D 

POSH'  computation 


This  computation  will  show  that  multiplying  the  WT  by  approximately  2  gives 
the  same  POSH  results  as  decreasing  the  POSH  offset  by  20.  Start  with  the  original 
POSH  equation: 


POSH  =  29  In 


SHI\ 
WT J 


+  50 


Now,  multiply  the  WT  parameter  by  2  and  separate  the  variables: 


POSH  =  29  In  SHI  -  29  ln(2  WT)  +  50 

Further  separate  the  terms: 


POSH  =  29  In  SHI  -  29  In  2  -  29  In  WT  +  50 


Combining  like  terms  gives: 


or 


POSH'  =  29  In 
POSH'  =  29  In 


rSHI > 
kWT; 
r SHI\ 
v  WT  j 


+ 

+ 


50-20.1 

29.9 


This  gives  a  POSH1  equation  very  similar  to  that  of  Eq.  7  in  Chapter  IV. 


